Phase diagram and large deviations in the free-energy of mean-field spin-glasses 
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We consider the probability distribution of large deviations in the spin-glass free energy for the 
Sherrington-Kirkpatrick mean field model, i.e. the exponentially small probability of finding a 
system with intensive free energy smaller than the most likely one. This result is obtained by 
computing $(n, T) = TZ"/n, i.e. the average value of the partition function to the power n as a 
function of n. We study in full details the phase diagram of ^(n, T) in the {n, T) plane computing in 
particular the stability of the replica-symmetric solution. At low temperatures we compute $(n, T) 
in series of n and r = Tc — T at high orders using the standard hierarchical ansatz and confirm earlier 
findings on the 0{n^) scaling. We prove that the 0{nP) scaling is valid at all orders and obtain an 
exact expression for the coefficient in term of the function q{x). Resumming the series we obtain 
the large deviations probability at all temperatures. At zero temperature the analytical prediction 
displays a remarkable quantitative agreement with the numerical data. A similar computation for 
the simpler spherical model is also performed and the connection between large and small deviations 
is discussed. 



I. INTRODUCTION 

The theory of disordered systems is mainly concerned 
with predictions regarding the most likely behavior, but 
it is also interesting to develop techniques to compute the 
probability distribution of rare events, i.e. the probabil- 
ity of finding systems that have properties different from 
the typical ones. There are various motivations: 

• We may have a special interest in those systems 
with a behavior different from the most likely one; 
for example in constraint optimization problems, 
in the region where it is impossible to satisfy all 
the constraints in the most likely system, there is a 
great interest in computing the properties of those 
rare systems where we can find a configuration that 
satisfies all the constraints [ij. 

• The properties of large fluctuations may be related 
to other more interesting properties of the system. 
For example given an intensive quantity Aj that 
depends on the system J of size N, in the large 
deviation region for large N we usually have that 
Pn{A) « exp{~NL{A)). It is quite common that 
there are relations among the behavior of Pn{A) 
in the region where the probability remains finite 
when TV goes to infinity and the behavior of L{A) 
near the point where L{A) — 0. In other cases [2| 
the techniques used to compute large deviations are 
the same used to compute other important quan- 
tities like (in finite dimensional spin glasses) the 
typical difference of the energy with periodic and 
antiperiodic boundary conditions. Besides sample 
to sample fluctuations have been recently shown to 
be related to chaos in spin glasses @ . 

• We notice also that the comparison between ana- 
lytic predictions in the large deviations region and 
numerical or experimental data could be important 



as a clear-cut test of the theoretical approach used 
to compute the most likely properties. 

Unfortunately even in the simplest non-trivial case, i.e. 
the Sherrington-Kirkpatrick (SK) infinite range model for 
spin glasses, there is no consensus on the results of such 
a computation. Everybody agrees that as a first step we 
need to compute in the large N limit the thermodynamic 
function 



Hn,P) = -^\nZj{f3r , (1) 

where different systems (or samples) are labeled by J, 
Zj{(3) is the partition function and the bar denotes the 
average over different disordered samples. It is well 
known that the probability of large deviations is related 
to the function $(n,/3). Indeed 



exp(-/3n7V$(n,/3)) = Zj(/3)" = exp(-niV/3/j(/?)) , 

(2) 

where /j is the system-dependent free energy per spin. 
The region of positive n corresponds to fiuctuations 
where the free energy is smaller than the typical one 
and the region on negative n corresponds to fluctuations 
where the free energy is larger than the typical one. 

There is a disagreement in the literature on the strat- 
egy we should follow to compute <I>(n, /3). In the n ^ 
limit the computation can be done using the broken 
replica symmetry ansatz (that is known to give the ex- 
act result), where it coincides with the most likely free 
energy <I>(0,/3) = ftyp or equivalently with the average 
equilibrium free energy/e, = ftyp. 

For n > Kondor [J| in 1983 presented a first com- 
putation of $(n,/3) in the region near using the most 
natural ansatz for replica symmetry breaking (RSB) ob- 
taining in the region of positive n 

$(n,/3) = /typ + C5n5 + 0(n6) . (3) 
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The result of Kondor was surprising: in the general case 
all powers of n are present in the Taylor expansion of 
$(n,/3) and for most of the systems we have $(71,/?) = 
ftyp + Ain + O(ri^), that is the typical situations for a 
Gaussian distribution of the free energy. The absence of 
the powers from to is due to cancellations and it 
was not clear if they were present only near the critical 
temperature. This form of the large deviation function 
implies that the probability distribution for / near (and 
smaller than) ftyp is of the form 

PnU) « eM-Na^/dftvp " ffh , (4) 

where ag/s = 5/36~^/^|c5|~^/^. The above relationship 
is valid for a small negative value of the free energy dif- 
ference A/ = / — ftyp that remains finite in the ther- 
modynamic limit. However it has not been possible to 
test directly Kondor prediction because presently all nu- 
merical data concern the fluctuations of the ground state 
energy, i.e. the system is at zero temperature. Indeed 
at zero temperature the free energy coincides with the 
internal energy and the numerical data are cleaner due 
to the absence of thermal noise. Instead many efforts 
have been concentrated on the scaling of the small devia- 
tions of the free energy. Indeed based on Kondor 's result 
and a matching argument (see discussion below) it was 
suggested in Q that the small deviations from its mean 
of the free energy per spin scale as N~^/^. This predic- 
tion has been put to test in a series of numerical works 
d, 0, i, H, [la [H [11 and although all estimates are 
smaller than 5/6 nobody has claimed that this value is 
definitively ruled out. More recent results strongly indi- 
cate that the fluctuations of the internal energy per spin 
at finite temperature scale as 7V~^/^, thus confirming the 
exponent obtained from [l^. However it was difficult to 
test the theory in absence of a quantitative prediction 
(the only prediction being on the exponent, a quantity 
that it is rather difficult to measure in a reliable way). 
Furthermore, the potential <&(n, beta) is naturally related 
to large deviations while the matching argument connec- 
tion with the small deviation exponent is not rigorous, 
see discussion below. 

More recently a different RSB ansatz was proposed by 
Aspelmeier and Moore [13, [lB|, who found $(n) = ftyp] 
in their approach the probability of large deviations goes 
to zero faster than exp{—L{f)N) and the matching argu- 
ment cannot be used to infer the small deviations expo- 
nent. Indeed there is a general agreement that for neg- 
ative n $(n) = ftyp and Pwif) goes to zero faster than 
exp(— CiV) as soon / > ftyp. Recent results [131 show 
that in that region we have PN{f) « exp(— A^^L2(/)) 
were the function £2(7) can be computed through the 
replica method. This 0{N'^) scaling of the logarithm of 
the large deviations probability in the positive A/ region 
is also observed in the spherical model where it has been 
recently derived using random matrix theory . 

We have concentrated on large deviations in the region 
/ < /typ, that corresponds to positive n. We have fol- 
lowed Kondor's approach and extended his computation 



to all temperatures, including T = 0; in this way we have 
obtained an absolute prediction for the large deviations 
distribution. Comparing our analytic results with the 
numerical simulations performed at zero temperature we 
found a remarkable agreement. We have worked in per- 
turbation theory assuming small t — T — Tc and small n 
and used appropriate resummation techniques to extend 
the computation down to zero temperature. We have also 
verified analytically that the 0{n^) scaling holds at all 
orders in perturbation theory and obtained an exact re- 
lationship between the corresponding coefficient and the 
derivative at a: = of the standard q{x) function. 

We also found that the alternative approach [13, [3| 
that predicts $(n) = ftyp for both negative and positive 
values of n cannot be valid for large positive n and there 
are no compelling reasons for which it should be valid 
at fixed positive n when N goes to infinity. This is in 
agreement with the results coming from an exact anal- 
ysis: for positive values of n Talagrand [11 was able to 
show rigorously that Kondor's approach gives the correct 
results. 

The paper is organized as follows. In section [TT] we in- 
troduce the functionals and the saddle point equations. 
In section IIIII we discuss the Replica- Symmetric (RS) 
solution and its stability, we compute the DeAlmeida- 
Thoulcss line in the (rt, T) plane and we discuss the be- 
haviour of the sample complexity above the critical tem- 
perature. In section [IV] we discuss the sample complexity 
in the low temperature phase and compare it with the nu- 
merical data. In section|V]we present a similar treatment 
of the spherical model, which being RS is considerably 
simpler. In section IVTl we discuss the connection between 
large and small deviations. In the last section we give 
our conclusions. In appendix VK\ we report the power se- 
ries of $(71, T) up to the 18th order. In appendix [Bl we 
present an analytical argument to prove that the O(rt^) 
scaling is valid at all orders in perturbation theory and an 
exact relationship between the coefficient and the deriva- 
tive dq/ dx m X = Q. A brief report on these results has 
been given in (l9| . 

II. SAMPLE COMPLEXITY 

We define the large deviation function for the free en- 
ergy, L(/), (that we will call in the following the sample 
complexity because it is related to the number of samples 
with free energy equal to /) as the logarithm divided by 
N of the probability density of samples with free energy 
per spin / in the thermodynamic limit: 

For large N the majority of the samples has free en- 
ergy per spin equal to ftyp, and all other values have 
exponentially small probability. Consistently L{f) is less 
or equal than zero, the equality holding / = ftyp, i-e. 
L{ftyp) = 0. For some values of / it is possible that 
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L(f) = — cx), meaning that the probabihty of large devi- 
ations goes to zero faster than exponentiaUy with N . In 
the thermodynamic hmit the function $(n,/3) defined in 
eq. (HI) yields the Legendre transform of L{ f) [5|, indeed 
we have: 



- f3n^n) = -(3nf + L{f) 
where / is determined by the condition: 

dL 



df 



and equivalently we have: 

L{f) = Pnf - Pn^in) 
where (3n is determined by the condition: 



/ 



dn 



(6) 



(7) 



(8) 



(9) 



Note that at any finite N, is also the generating 

function of the cumulants of the distribution of the free 
energy. 

In the Sherrington-Kirkpatrick model at low temper- 
atures the replica symmetry is spontaneously broken for 
the generic system, i.e. in the n — > limit. One knows 
that at high positive values of n, replica symmetry is not 
broken ^20j . Therefore for positive n one must distinguish 
two regions in the T — n plane separated by the so called 
de Almeida Thouless (dAT) line, see fig. (H]). In the re- 
gion above the dAT line, the phase is replica-symmetric, 
while replica symmetry is broken below. 



III. THE REPLICA-SYMMETRIC PHASE 

In the Replica-Symmetric (RS) region the order pa- 
rameter is the overlap q. The corresponding value of the 
potential ^{n,q) is given by 



$(n,/3)M 



/3 



{l-2q+{l-n)q^) + 



/3n 



dy 



:e 2<! (2cosh/3?/)'' 



The overlap q can be computed by solving the equation 
d^{n,q)/dq = that yields: 



J e ^1 (cosh (3y)^ tanh^ (3y dy 
/ e 2, (cosh/3y)" dy 



(10) 



A. The High- Temperature Region 

The search for solutions with g 7^ of equation PT!]) at 
given T and n was done integrating numerically the r.h.s. 




PARA 



FIG. 1: Phase Diagram of $(n) in the {T,l3n) plane. In 
the paramagnetic phase the solution is RS with q — and 
^{n,l3) = -/3/4 - ln2//3. The dashed line /?nc(T) marks a 
first order transition to a RS spin-glass phase where q jumps 
from zero to a finite value qdT). The line nc{T) ends at the 
point (T = l,n — 2) where qc — 0. The vertical line from the 
point (1,0) to (1,2) marks a phase transition from a paramag- 
netic to a RS spin-glass phase with the parameter q changing 
continuously. The dAT line marks the region of stability of 
the RS spin-glass phase, below the line the phase is RSB. 
The value of PudAT diverges in the zero-temperature limit 
as /3n ~ -^Z— 2 ln[3(7r/2)^/2r], as a consequence the function 
L(Ae) at zero temperature is described by the RSB solution 
at any value of Ae. 



for different values of q and checking if the corresponding 
curve y = y{q) crosses the line y — q. 

In the high temperature region T > 1 the solution 
in the n — > limit is replica symmetric with q — Q. 
For small n there is no other solution, therefore $(n) — 
fas = -/3/4 - ln2//3. At n = n*(T) a new solution 
appears, but it has a value of ^RS,q^o{''^) larger than 
fiis and must be discarded {^{n) must be a continuous 
function of n) . As soon as n > n* (T) the solution bifur- 
cates into two solutions (?> (n) > g< (n) with the largest 
solution having a smaller value of <I>(n) than the other. 
Increasing n we cross the line n = nc{T) > n*{T) where 
^RS.q^in) = fas and the RS solution qy{n) becomes 
the physical solution for all n > nc{T). Summarizing 
the behaviour of n$(n) as a function of n in the high 
temperature phase is: 

n$(n) = nfns for n < Uc 

n $(n) = ricfBs + fRS.q.^n + 0(An)^ for n > Uc 

where An = n — Uc and we have omitted the depen- 
dence on the temperature of Jrs, n-c and fRS,q^ = 
dn^/dn\^^^+. At n ^ ric the order parameter jumps 
from zero to qc = qy{nc), the free energy turns out to be 
discontinuous too: fRS,q^ < Irs- Thus as a function of 
n n$(n) has a first order transition at n = riciT), see fig. 
([T|). This peculiar behaviour of <I>(n) refiects itself in the 



4 



following structure of L{f) as follows from eqs. (|6I7I8I9P : 



B. At the Critical Temperature 



L(/) = for / = fns 

L{f) = -oo for fRs,qa < f fns 

L{f) = ^Pn^ifns - fRS,,J + (3n,Af + OiAff for A/ 

where A/ = / - fRs,q,- 

To understand this double-peak behaviour of L{f) it 
can be useful to think of the Random-Energy-Model 
(REM) j22| ■ In the typical sample the number of energy 
levels with energy E is proportional to exp(A^ In 2 — A^E'^) 
and there are no levels outside the band (— Vln 2, Vln 2). 
In the high-temperature phase the energy of the typi- 
cal sample is given by the point where the derivative of 
(ln2 — E"^) is equal to /3, i.e. E — —(3/2 and the critical 
temperature is given by (3c = 2\/ln2. The free energy 
is given by F = — /3/4 — \n2/(3. Now in order to re- 
duce the energy of such a sample at fixed (3 < (3c one 
should modify the structure of the energy levels. One 
can see that any modification of the global shape of the 
distribution of the N energy levels has a prohibitive price 
with a probability 0(exp[— 2^]), nevertheless one could 
instead pull a level out of the band with a cost in prob- 
ability 0(exp[iV]). Normally the energy of the lowest 
level is — Vln 2 which is larger than the free energy of 
the levels with energy —(3/2. Thus a small modification 
of the lowest level will not have any effect on the total 
free energy that will be still dominated by the levels with 
E = —(3/2. Only when the energy of the lowest level 
becomes smaller than —(3/4 — In 2//? the thermodynamic 
of the sample is dominated by the lowest state and the 
energy jumps abruptly from E = —(3/2 to a lower value 
E = -(3/A-\n2/(3. 

Coming back to the SK model we observe the following 
behaviour of nc{T) and qc{T) approaching the critical 
temperature: 



On the line T = 1 we have $(n,Tc) = fRS,q=o[Tc) = 
— 1/4 -I- In 2 for n < 2 while for positive An = n — 2 the 
^ golution is still RS but with a non zero value of q; at 
~ leading orders we have: 



lim UciT) = lim n*(T) ^ 2 



hm qc(T) = |un g< (T) = 



(11) 



(12) 



Therefore at the point (T — Tc,n = 2) the discontinuity 
in the free energy vanishes and it represents the end-point 
of a the line nc{T) of first order phase transitions. More 
precisely near the critical point T = Tc and n = 2 we 
have at leading order in t ~ Tc — T < 0: 



n 



■(T) 



T 

3 



1/2 



UciT) 
9c (T) 

fRS,q^{T) - fRsiT) 



Irs + 

2 + (-3t)'/' 



1/2 



2 (-3r) 



q 



3An + 0(An2) 
9 



$(n,l) = /fls.,=o - ^An* + 0(An 



(13) 
(14) 



Note that at the critical temperature the range where the 
sample complexity is finite touches /hs, and L(A/) = 
2A/ + 0(A/^). This behaviour is interesting in connec- 
tion with the problem of the small deviations of the free 
energy as we will discuss below. 



C. The low- Temperature Replica-Symmetric 
Region 

On the straight line that connects the point (T — 
1, n = 0) and (T = 1, n = 2) the potential <I>(n) has also a 
phase transition, see fig. ([T]). On the right of this line we 
have g = and $(n,T) = fRS,q=o{T) = -(3/4 + ln2/(3. 
On the left the solution is still RS (except for n = 0, see 
below) and the parameter q has a continuous transition. 
At the leading order in r = — T we have (for n < 2 
and < T < 2 - n): 



q{n,T) 



2-n 
^{n,T) = fB.s,q=o 



2(1-") _3 , ^,4 



3(n- 2)' 



(15) 

t^ + 0(t*) (16) 



Note that the physical $(n,r) is equal to fRs,q=o{T) = 
—(3 /A — ln2//3 at any temperature for n = 1, while it is 
larger for n < 1 and smaller for n > 1. 

The above expansion in powers of t breaks down at 
n = 2. On the ti = 2 line the RS solution satisfies the 
exact equation: 



q = tanh (3q 



(17) 



this leads to the following behaviour at leading order in 

T 

q = ^/67+0(r3/2) (18) 
$(2,T) - /^s^^^o - + 0(r3) (19) 



On the other hand near n = Q the RS solution is in- 
consistent, indeed for convexity the function n$(n) must 
have a negative second derivative with respect to n but 
this condition fails at any r for sufficiently small values 
of n. According to eq. we have for small positive r: 



(9^n$(n) 



(20) 



5 



thus this quantity goes to zero for small n and we have 
to take care of the 0(t'^) term. Taking into account the 
0{t'^) term and expanding in powers of n we have: 



$fls(n,T-)-$i?s(0,r) 



24 



- — +0(r,n,5) (21) 



from the above equation we see that for n < 2t/3 + 0(t)^ 
the second derivative of the n^{n) would be positive and 
the solution inconsistent. Thus in the (/3n, T) plane there 
is a line nconv{T) — 2t/3 + 0{t)^ below which the RS 
solution is inconsistent for convexity reasons and cannot 
be the correct one. In the following we will see that 
actually the RS solution becomes unstable and should 
be discarded below a line ndAT{T) = 4t/3 + 0(t)^ which 
is above the line nconv{T). 

In the low temperature phase the replica solution is 
unstable at small values of n as the replicon eigenvalue 
becomes negative [2l|. Similarly to the stability of the 
RS solution in the magnetic- field/Temperature plane, 
in the (n, T) plane the region of stability is above the 
deAlmeida-Thouless (dAT) line that is specified by the 
condition: 




FIG. 2: The potential f&^T) vs. temperature at equi- 
librium {n = 0) (solid) ;24:] and on the dAT line (n = 
ndAT{T)) (dashed), for small t = Tc — T the difference is 
$(0,r)-$(ndAT(r),T) ^ t^/135 + 0{t^). The Potential on 
the dAT line diverges as —f3ndAT{T)/4: — In 2 / jSudAT at low 
temperatures. 



J e 2, (cosh /3y)"(l — tanh^ f3y)^ dy 



(22) 



J e ^1 (cosh/3j/)" dy 
For small r = 1 — T the value of n on the dAT line is 



while in the zero temperature limit 
we have: 



(23) 

q goes to unity and 



ndAriT) = TW-21n 



1/2 



T 



(24) 



Note that UdAT vanishes in the zero-temperature limit 
but in the (T, nP) plane the dAT line never touches the 
T ~ line and the sample complexity L{e) at T = is 
always in the RSB phase, see fig ([T]). On the other hand 
this show that at any fixed n > the system exit the 
RSB phase at low enough temperature and the solution 
is always RS at zero temperature. In figure ([2|) we plot 
the potential $(n,T) on the dAT line, it goes to minus 
infinity at low temperature as: 



$(«rfAT(T),r) 



(3ndAT{T) 



In 2 



(3ndAT{T) 



(25) 



Note that the second term gives a vanishing correction 
that can be rather large at finite temperature. 



D. The Large n Limit 

At any finite temperature ^{n) is described by the RS 
solution at large values of n. Both above and below the 



critical temperature, the behaviour of $(n) for large val- 
ues of n is $(n) = -/3n/4 - ln2/(/3n) -I- 0(e-2/3»). This 
leads to L{f) = — /^-|-ln 2-|-o(l) for large negative /, note 
that this is the same behaviour of the Random-Energy- 
Model (REM)!!. 



IV. THE REPLICA-SYMMETRY-BREAKING 
PHASE 



Below the dAT line n < ndAT{T) we must break the 
replica symmetry. As we can see in fig. ^ $(n) on the 
dAT line is smaller than the most likely free energy {ftyp) 
that is $(n) at n = at the same temperature, in partic- 
ular ^{ndAx) diverges as -f3ndAT{T)/4:-\n2//3ndAT at 
low temperatures while for small t — Tc ~ T the differ- 
ence is $(0, T) - <^>{ndATiT),T) = tV135 + 0[t^). On 
the other hand being a convex function $(n) must be 
continuous, therefore we must look for a free energy that 
shows some dependence on n also below the dAT line and 
the one suggested by Kondor is the most natural one. 

We recall that in Kondor's approach for n < 
ndAT{T) < 1 one introduces a function q{x) defined for 
n < X < 1 that describes the breaking of replica sym- 
metry in the low temperature phase. A functional Fn[q] 
is obtained such that ^(ji) = maxq Fn[q]. The function 
q{x) that maximizes Fn[q] can be found by solving the 
stationarity equation SF/6q{x) = 0. This generalizes the 
standard approach that is proved to give the correct value 
of $(71) in the n ^ Q limit. 

The form of the free energy functional is the usual 
one [2l|, the only difference being that all functions are 
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defined in the interval n < x < 1: 



F^[q{x)] = -f (l - 2g(l) + q\x) dx 



fin 7_oo V^thtH V 2(7(n) 



the function /(x, obeys the following equation: 



(26) 



where dots and primes mean respectively derivatives with 
respect to x and y. The initial condition is 



= ■^log2cosh/3y 



(27) 



The above functional has to be extremized with respect 
to the function q{x). A set of variational equations can 
be obtained introducing Lagrange multiplier P{x,y) to 
enforce equations (^5)1 and (P7|) [24 1251] , the resulting 
equations are: 



q{x) 



m 



m 



P{x,y)m'^{x,y)dy 



-(to" + 2xP mm') 



P=1{P" -2x[3{mP)') 



(28) 



(29) 



(30) 



These are the same equations of the standard n case, 
the only difference is in the initial condition for P{x,y) 
that reads: 



P(0, y)=c exp 



2q{n) 



/3nf{n,y) 



(31) 



where c is a normalization constant in order to have 
/ P{x,y)dy = 1. Since $(n) is extremized with respect 
to q{x), the conjugate variable / can be obtained as 
the derivative of the n^{n,q{x)) evaluated at the sad- 
dle point: 

/ = -^ [l-2q{l) + q\x)dx-nq\n)^-{f{n,y)) 

(32) 

where the square brackets represent average with respect 
to the measure d^ = exp(— (?/ — h)'^/2q{n) + Pnf{n, y)). 

We have solved the RSB equations and computed q{x) 
and $(n, f3) as power series of n and r = 1 — T [23]; the 
power series of $(rt) up to 18th order is reported in the 
appendix. Kondor originally used the so-called truncated 
model valid near the critical temperature and he found 



we have confirmed that the lowest power of n in the ex- 
pansion of <I>(n) is and that there is also no term. 
We have verified by an expansion in powers of n at fixed 
temperature that the first term in $(n) is of 0{n^) at all 
temperatures as follows from an analytic argument pre- 
sented in the appendices. This result is also related to 
the behaviour of the free energy functional with increas- 
ing number of RSB steps [28| . An alternative argument 
can be done using the expansion of the replicated free 
energy functional F\Qab\ [2ll ] in powers of the n x n ma- 
trix Qab, at least for powers less than 10, where one can 
use the explicite form of the terms. The expressions be- 
come more complex when the power of Q become larger 
or equal to 10. 

For negative n the saddle point of the $(«) is the stan- 
dard q{x) corresponding to n = 0, thus $(n) = f^q for 
n < 23]. The corresponding sample complexity as a 
function of A/ = / — ftyp reads: 

L{f) = -oo for A/ > 

L(/) = ae/slA/l^/s + OdA/lS/S) for A/ < 

Where ag/5 — — 5/3]c5]^^/^6^^/^ and C5 is the coefficient 
of in the expansion of $(?i). The function q{x) has a 
small plateau from n to some value Xc- For a;c < a; < 1 
q{x) has the usual shape, more precisely deviations of 
q{x) from qfree{x), i.e. the solution corresponding to 
n = 0, are O(n^) in this region. We note that q{x) is 
always continuous at the end-point of the first plateau 
Xc. At the leading order in n we have: 

3 3 
q{x) = -nq{Q) + 0{n'^) for n < x < Xc = -n + 0{n'^) 

q{x) — qfree{x) + 0{n^) for Xc < x < 1 

where q{0) is the derivative of qfree{x) in a; = 0. Note 
that the above expression are valid at all temperatures 
since they have been obtained from the fixed temperature 
expansion in powers of n reported in the appendices. We 
have also verified directly these features of q{x) at the 
order to which we computed the expansion in n and r. 

It is interesting to note that from the third order on, 
all derivatives of $(n) (with respect to ri,T and both) are 
discontinuous on the dAT line i.e. the transition is third 
order. This is the same behaviour of the free energy on 
the dAT line in the [h,T) plane [2I]. 

When (3 ^ 00 the complexity L{f) goes to a well- 
defined limit therefore from eq. ^ $(n) is actually a 
function of (3n [s^] , as a consequence the coefficient Ca 
of in the power series of ^{n) diverges as P"" in the 
zero temperature limit. 

The series in power of r of C5 (the coefficient in 
^(n)) can be used to obtain its behaviour in the whole 
low temperature phase provided one uses the information 
that C5 ~ (3^ in the zero-temperature limit. In figure [3] 
we plot various Fade approximants obtained from the 
series of c^l3~^ . From the Fade approximants of C5/3~^ 



that $(n) = feq - 9n^/5120 4]. At all orders considered and 07/?"" we estimate C5 ~ -0.0060(2) (3^ near T = 



A/ 
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T FIG. 4: Sample complexity -L(A/) at T = .7, obtained 

through an (8, 5) Pade approximant to the parametric power 
FIG. 3: Four Fade approximants ( P(3,3), P(3,4), P(4,3), series in r. The dot marks the RSB-RS transition at A/ = 
P(4, 4) ) of C5/3 ^ as a function of the temperature, cs/S ^ is _i g4 ^ 10~^ 
equal to -9/5120 at T = 1. 



and C7 ~ -0. 0150(5)^0'^ in the SK model. Alternatively 
one can use the following exact relationship valid at all 
temperatures (see appendix [BJ : 

C5 = -^l3'{Tmf (33) 

where q{Q) is the derivative in a; = of the usual function 
q{x) for 71 ^ 0. The zero-temperature limit of Tq[Q) is fi- 
nite and was obtained resumming its expansion in powers 
of T as .743(2) in [2J]. Recently a more precise estimate 
.743368 has been obtained working directly at zero tem- 
perature [28|, this gives Yanp^oa cs/?"^ = —0.0057766. 

The zero temperature complexity for negative Ae then 
reads: 

L(Ae) = -1.63250 jAej^/^ + 3.1(1) |Ae|*^/^ + 0(Ae^/^) 

The second term however yields a big correction to the 
first one, indeed: i) the exponents of the series grow 
slowly (as (6 -I- ?)/5, i = 0,2,3,..., note that there is 
no term in $(n)) and ii) the coefficients of the series 
grow quickly with order, actually we expect the series to 
be asymptotic as is usually the case in this context [2^ . 
Therefore in order to have a good control on L{Ae) we 
have adopted a method previously used in [11] to obtain 
q{x,T) from its series in powers of x and t. We have 
transformed the series of L{Af) in powers of A/ and r 
in a power series of just r by setting A/ = {-^s^ + \t'^)c 
with c a parameter in the range [0, 1]. The correspond- 
ing series in powers of r were resummed for any given c 
through Pade approximants obtaining the curve L(Ae) 
in parametric form. 

In fig. (jl]) we have plotted the function L{Af) at tem- 
perature T — .7 obtained by resumming the series of 
L(A/(t, c)))/t^ by means of a Pade approximant of or- 
der (8, 5) (we have used the series of $(n) to 18th order 
reported in the appendix). 

In figure (O we plot the sample complexity L(Ae) at 
zero temperature. In the range of energy differences 



considered the deviations from the values yielded by 
i(Ae) = -1.63250 \Ae\^/^ are no larger than 1%, this 
support the goodness of both estimates since they were 
obtained by different resummation schemes. 

By resumming the series of $(n), (see below) we have 
been able to obtain the sample complexity in the whole 
low-temperature phase and for finite L(f), in figure [5] 
we compare the sample complexity with the numerical 
data at zero temperature of Ref. l§\ finding a very good 
agreement. For each system size N we have plotted 
Ln = ln(P(Aejv)/iV5/6)/Ar with Ae^v = e — Cat (the 
average energy at size N), we have used this definition 
so that L goes to a constant for Acn = 0. The errors 
on Ln have been computed through error propagation 
and single events have been discarded to reduce the er- 
ror. The quantitative agreement of the numerical with 
the theory is quite good. 

In figure [5] we have also plotted the numerical com- 
plexity (from Ref. Q) as a function of the absolute value 
of the energy for different sample sizes at zero tempera- 
ture. The data have been shifted vertically by an amount 
Ajv so that the complexity vanishes at the typical energy 
Etyp = —.7633. Since this certainly holds in the thermo- 
dynamic limit An goes to zero at large N. Note that 
for E < Etyp the numerical data approach the theoreti- 
cal prediction from below, this rules out the alternative 
prediction of Ref. [13] that yields L(Ae) — —00. 

V. ON THE MEAN-FIELD SPHERICAL 
SPIN-GLASS MODEL 

The RS expression of $(n, T) of the spherical model is 
$(n,T)[.,g] = -^-|-i^/3g^ + 
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FIG. 5: Comparison between the numerical and analyti- 
cal sample complexity at zero temperature, see text. The 
data are those of Ref. [^. The sample complexity was 
obtained through an (8, 5) Fade approximant to the para- 
metric power series in r, the deviations from the expression 
L(Ae) = -1.63250 \I^efl^ are less than 1% in this range of 
energy differences. 



at some temperature dependent value n = nc(T) there is 
a first order transition. The line nc{T) ends on the point 
{T — l,n — 2), around that point it goes as ndr) ~ 
2 -I- 3-y/— T for negative t = T^ — T . The straight line from 
the point (T = l,n = 0) to the point (T = l,n = 2) 
divides the paramagnetic from the spin-glass {q ^ 0), 
at variance with the nc{T) line, the parameter q varies 
continuously upon crossing this line. At all temperature 
the physical value of $(n,T) is smaller than expression 
([55)1 for 71 > 1 while it is larger or equal to it for n < 1. 

Below the critical temperature only the spin-glass 
phase is present. The main difference with respect to 
the SK model is the absence of the dAT line in the phase 
diagram, meaning that the RS solution remains correct 
in the limit n — > (s^l- The expression of $(n,T) for 
T < 1 at small values of n is: 



ci>(n,/?) 



1 



1 



HP/2] {(3 ~ If 
2(3 24/3 



■n^ + 0{n^) (36) 



Note that again the linear term in n is missing and the 
sample complexity is non-Gaussian: 
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FIG. 6: Plot of the numerical complexity (from Ref. [Si]) as 
a function of the absolute value of the energy for different 
sample sizes at zero temperature. The data have been shifted 
vertically by an amount A at so that the complexity vanishes 
at the typical energy Etyp = —.7633. 



1 ^ 

_ hn[z + !±-q(l - n)] + (n 



l)ln[z+^g]) (34) 



The above expression has to be extremized with respect 
the parameters q and z. The phase diagram of the 
^{n,T) in the {T,Pn) plane is qualitatively similar to 
that of the SK model, see fig. (jT]) , except for the ab- 
sence of the dAT line. In particular above Tc = 1 the 
solution is paramagnetic {q = 0) for small n and does 
not depend on n, 



$(n,/3) 



1 

'2^ 



4 



In 2 



(35) 



L{f) = -oo 

!(/) ^ -i^ 



for A/ > 



|A/| 



3/2 



(3-1 



3/2 



for A/ < 0(37) 



At zero temperature the energy of the model is equal 
to minus the largest eigenvalue of a Gaussian random 
matrix 



(38) 



The small deviation distribution of the largest eigenvalue 
of a Gaussian random matrix is given by the Tracy- 
Widom law Fi (x) [3l| in terms of the rescaled variable 
X = V2{Xmax - ^/2N)N^/^. The behaviour of Fi{x) for 
X +00 is: 



lni^i(2;) 



^3/2 



4\/2 



|Ae|3/2^ (39) 



where we have used x = 27V^/'^|Ae| as follows from eq. 
([55)) . Thus eq. ([57)1 and (|VI[) give the same prediction 
and there is perfect matching between small and large 
deviations. A similar matching has been also observed 

for positive deviations Ae whose probability scales as 
exp[0(7V2)], [g. 



VI. ON SMALL DEVIATIONS 

In this section we discuss the connection between small 
deviations and large deviations of the free energy. The 
function <I>(n) is the natural object to describe the large 
deviations of / from its typical value. In it was argued 
that it provides also information on the small deviations 
of the free energy arguing that they scale as N"^^^. 



9 



The probability distribution of the free energy per spin 
Pnif) is concentrated near the typical free energy ftyp 
in the large N limit. The small deviations corresponds to 
values of the free energy difference that have a finite prob- 
ability PnH) — 0(1) to be observed in the thermody- 
namic limit, that is a region near ftyp that shrinks to zero 
in the thermodynamic limit. On the other hand the large 
deviations corresponds to the exponentially small tails of 
PN{f) corresponding to 0(1) values of the free energy 
difference. Thus in principle small and large deviations 
are fairly different objects and it may seems strange that 
one can determine the scaling of the peak from a large 
deviation calculation. 

Typically the probability of the small deviations scales 
as limAr^oo-PAr(/) p{{f - In)/!^-"-) where p{x) is a 
scaling function that does not depend on iV, a is some 
positive exponent and /at is the A'^-dependent mean value 
of the free energy that converges to ftyp as Jn = ftyp + 
for some positive b. 

The argument that connects large and small deviations 
is not rigorous and relies on the assumption that there is 
smooth matching between the behaviour of the peak of 
Pjv(/) and the left tail corresponding to positive values 
of n. Under this assumption one argues that the region of 
the peak corresponds to values of the free energy differ- 
ence {ftyp — f) such that the large deviation expression 
(HI) is finite, this happens for {ftyp - f) = 0{N-^/^) 
and leads to the aforementioned prediction a = 5/6. In 
other words the matching argument corresponds to the 
assumption that the function p{x) provides a good de- 
scription of the distribution of the free energies up to 
free energies differences A/ = 0(1), i.e. far beyond its 
natural range of validity A/ — 0{N^°-). 

It is interesting to note that in order to characterize 
the small deviations one should take the n — > limit first 
and then the iV — > oo limit while the two limits have to 
be inverted to obtain the large deviations. 

In the following we discuss what kind of quantitative 
information can be extracted from the thermodynamic 
limit of $(n) under the assumption that there is a smooth 
matching between small and large deviations. 

The starting observation is that at any finite value of 
N, n$(n) is the generating function of the cumulants 
of the distribution of F. If in the thermodynamic limit 
— ftyp + Ca n°-N^^ one would consider the variable 
X = (F-Fjv)/^'^"'')/(°+^\ where F/v is the average free 
energy at size iV, and claim that the a — th cumulant 
of its distribution function f{x) is finite while all higher 
cumulants are zero. In particular in the high temperature 
phase we have — ftyp + Ca nN^^ and we would say 
that the fluctuations of F around its average value F/v are 
normal with finite variance because all cumulants higher 
than the second vanish. 

Extending this argument to the low temperature case 
one could say that all the cumulants greater than the 
sixth of the variable x — {F — Fn)/N^^^ vanish. This 
conclusion however is wrong because the function (f>(n) 
in the low temperature phase has a first order phase tran- 



sition at n = and the two limit N ^ oo and n ^ 
cannot be exchanged in computing derivatives of $(n). 
The only exception is the zero-th derivative <I>(0) (the av- 
erage free energy). This can be also understood noticing 
that by fixing n and taking the limit N oo the actual 
value of X goes to infinity, therefore there is in principle 
no way to get information on the small- a; region once the 
thermodynamic limit has been taken. However under the 
assumption of smooth matching between small and large 
deviations it is also natural to assume that the small de- 
viations of the free energy behave for large negative x 
as: 

p(x)^exp(a6/5|a;|6/5) (40) 

In this context it is instructive to consider the REM at 
zero temperature [12]. The complexity as a function of 
Ae = e — Vln 2 behaves as L{Ae) = — /3cAe for Ae < 
(with Pc — 2Vln2) while L = —oo for positive Ae. Using 
the matching argument between small and large devia- 
tions we would conclude that the extensive energy has 
finite variance and that the behaviour of the rescaled vari- 
able X — Af(e— ejv) for x negative and large is exp{—(3cx), 
this prediction is consistent with the known fact that the 
small deviations obey the Gumbel law. However in order 
to recover the full Gumbel distribution of small devia- 
tions we should take the n limit first. Note also that 
the deviations of E from its mean En (which is 0(1) in 
this case) has nothing to do with the deviations of En 
from its thermodynamic limit which is 0(ln N) in the 

REM m. 

Another example of matching between small and large 
deviations is provided by the spherical model as discussed 
in the previous section. 

On the other hand the very same SK model at the 
critical temperature seems to provide an example of the 
failure of the matching argument. Indeed above the criti- 
cal temperature the sample complexity is — oo for A/ 7^ 
and has no role in the finite-size fiuctuations of the free 
energy which instead are controlled by corrections to the 
q = solutions 0, [l^. The free energy F has a fi- 
nite variance that diverges for T ^ Tc. Therefore it is 
expected that the free energy variance diverges with N 
at the critical temperature. Indeed extending the com- 
putation of Ref. [IBl Aspelmeier has recently argued 
that the variance diverges logarithmically with N 0]. 
On the other hand at the critical temperature the range 
where the sample complexity is finite touches /as, and 
L(A/) = 2A/ -t- 0(A/2); therefore applying the match- 
ing argument one would wrongly conclude that the be- 
haviour of the distribution of F is exp(2F) for large neg- 
ative F and that its variance remains finite. However we 
believe that the matching argument is correct for the SK 
model below the critical temperature and that its fail- 
ure at the critical temperature can be explained noticing 
that there is a phase transition in the (n, T) plane on the 
straight line connecting the point (0, 1) and (2, 0) and all 
the eigenvalues of $[(5°^] vanish on this line. 
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VII. CONCLUSIONS 

We have computed the function <i>(n, T) of the SK 
model using the hierarchical ansatz and discussed its be- 
haviour in detail both above and below the DeAlmeida- 
Thouless line in the (n, T) plane. In particular in the 
low-temperature phase we have confirmed at all orders 
Kondor's early result [J] on the 0{n^) scaling. The ana- 
lytical argument provides an exact relationship between 
the coefficient of the term and the x — derivative of 
the the standard function q{x). We note that the same 
approach provides a similar exact relationship between 
g(0) and the 0(/i^°/^) term in the equilibrium free en- 
ergy in presence of a magnetic field h. By resumming 



the series we have been able to obtain for the first time 
the sample complexity at zero temperature. Existing nu- 
merical data display a remarkable agreement with this 
prediction. We mention that the presence of a magnetic 
field reintroduces a 0{n^) dependence in n^(n) leading 
back to Gaussian fluctuations of the free energy. 

APPENDIX A: POWER SERIES OF $(n) 

In this appendix we report the power series of <I>(n) of 
the SK model in the low temperature phase up to the 
18th order in n and r = 1 — T. At all order in r the 
smallest power of n is and there is no term. 
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APPENDIX B: THE VARIATIONAL FREE 
ENERGY AND THE PLATEAU OF q{x) 

In this appendix we consider the effect of a small 
plateau in the function q(x) at small values of x. Such 
a plateau in the function q{x) is present in two notable 
cases: i) when a magnetic field is present and ii) when the 
parameter n is finite. In both cases the perturbative so- 
lution in power series near the critical temperature shows 
that the dependence of the plateau and of the free energy 
on the small perturbations (i.e. the value of n or h) is 
anomalous: i) we have seen in the previous sections that 
the behaviour of at = is <I>(n) = feq + 0{n^) 

and ii) it is well-known that the fourth derivate of the 
free energy (n = 0) with respect to the field is divergent 
f{h) = feg{0) - + 0{h^°^^) HH, m. The origin of 
this can be traced back to the fact that the variation of 
the free energy in presence of a small plateau is a fifth 
order effect. Here we show that this can be proved at all 
temperatures. 

We also note that the fact that the presence of a small 
plateau of eight qo gives an O((7o) correction to the free 
energy provides further insight into one of the earliest 
observations on the RSB solution [2^, namely the fact 
that the corrections to the free energy due to using a 
finite number K of RSB steps decrease like K^'^. Indeed 
the function q{x) in this case is a set of K small plateaus 
with 0{K'^^) differences from the true solution and it is 
natural to expect the total free energy correction to be 
0{K-^) X K. 

We consider the variation of the free energy functional 
^{n,h,q{x)) as a function of n, h and qo under the 
assumption that q{x) is unperturbed for x > Xpiateau 
(where Xpiateau is such that q{xpiateau) = qo) while 



q{x) = qo for n < x < Xpiateau- The main result of this 
appendix is the following expression valid at any temper- 
ature: 

h?nBqo n'^B^q^ 

$(n, h, qo) - $(0, 0, 0) - -— - + 

2 2 D 



15g2(0) 4(j(0) 24g(0) 



sixth order terms. (Bl) 



Where q{0) is the derivative of the Parisi solution q{x) 
in X = for h = n = 0. The meaning of the last term is 
that this expression is valid at all temperature but at the 
lowest orders in h, n and qo- The first term —h'^/2 yields 
the known result that the zero-magnetic-field suscepti- 
bility is equal to one in the whole spin-glass phase while 
the remaining terms are fifth order in qo in the sense that 
n = O{qo) and = 0{q^). 

Extremizing the above expression with respect to go at 
h — we get: 



Imio) 



o{n) 



(B2) 



and |3S 



'^{n) = fe, — M)5(Tq(0))'^ + Oin') (B3) 

Conversely, extremizing with respect to qo at n ^ we 
get: 

^ ^3Tmy\m + oih'^') (B4) 
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and 

fih) = /(O) - — + —( -\-] ^'^ + o(/ii"/3) 
' ' 2 20 \^Tq{id) ) ^ y I 

(B5) 

Thus the anomalous behaviour of the free energy at small 
n or h found near the critical temperature holds true at 



all orders, and the coefficients of the terms O(n^) and 
0{h^°/^) in the above expressions depend on the tem- 
perature only through the term Tq{0), that has a finite 
limit at zero temperature. 

The quantity q{0) can be computed in power series 
near the critical temperature ^2?! and reads: 



1 3r „ . „ 4 336 . 136884 979779 71633011 1077802999 r 
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18770216489 r" 68028264769963x^2 1136615361900763 r^^ 1084041597207443333 tI"* 
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1162702256772757485034973381 t^^ 193682918656993987102843106053 r^^ 



231567336000 2199889692000 



+ Oir'") (B6) 



This expression can be resummed through Pade approx- 
imants '24^ in order to obtain quantitative predictions in 
the whole low-temperature phase, e.g. in the zero tem- 
perature limit we have [24]: 

lim Tq(0) = 0.743 ± 0.002 (B7) 

A more precise estimate limT^o Tq{0) = .743368 was ob- 
tained recently working directly at T = in [2^. From 
the power series expression of q{0) the corresponding 
power series of the coefhcients of the 0{n^) and 0{h^°/^) 
terms in the expressions (jB3|) and jBSj can be computed 
and they are in full agreement with the corresponding 
expressions computed through the power series solution 
of the variational equations, i.e. eq. (jAl[) above and 
eq.(lO) of Ref. 

The variational expression eq. (|B1[) can be obtained 
by computing the function f(x,y) as a power series of x 



around x = using the evolution equations (|26p up to the 
fifth order. To obtain the result we need to use the fact 
that f(x,y) is an even function of y and most impor- 
tantly the following three exact statements concerning 
the functions q{x) and f{x,y) computed at n = /i = 
(see Ref. ^2^] for their derivation): 

m = (B8) 



/("^2)(o^O) = 1 (B9) 



/(''^')(o,o) = - 
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